Switchable Pancharatnam-Berry phase grating stack

ABSTRACT

Various embodiments set forth a foveated display system and components thereof. The foveated display system includes a peripheral display module disposed in series with a foveal display module. The peripheral display module is configured to generate low-resolution, large field of view imagery for a user&#39;s peripheral vision. The foveal display module is configured to perform foveated rendering in which high-resolution imagery is focused towards a foveal region of the user&#39;s eye gaze. The peripheral display module may include a diffuser that is disposed within a pancake lens, which is a relatively compact design. The foveal display module may include a Pancharatnam-Berry Phase grating stack that increases the steering range of a beam-steering device such that a virtual image can be steered to cover an entire field of view visible to the user&#39;s eye.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to patent applications entitled “FoveatedDisplay System” and “Pancake Lens Including Diffuser”, which areassigned to the same assignee of this application and filed on the sameday as this application, and which are herein incorporated by referencein their entireties.

BACKGROUND Field of the Various Embodiments

Embodiments of this disclosure relate generally to optical systems and,more specifically, to a foveated display system.

Description of the Related Art

Artificial reality systems display content that may include completelygenerated content or generated content combined with captured (e.g.,real-world) content. A realistic display should account for what a usersees in his or her peripheral vision, as well as the high-acuity visionproduced by the fovea centralis (also referred to herein as the “fovea”)located in the back of the user's eyes. For some artificial realitysystems, such as head-mounted display (HMD) systems, a small form factorand light design are also desirable. Designing such artificial realitysystems has proven to be difficult.

SUMMARY

One embodiment of the present disclosure sets forth an optical system.The optical system includes two Pancharatnam-Berry Phase (PBP) gratings.The optical system further includes a switchable half-wave platedisposed between the PBP gratings.

Another embodiment of the present disclosure sets forth a displaysystem. The display system includes a light source. The display systemfurther includes an optical stack that includes a plurality ofPancharatnam-Berry Phase (PBP) gratings. The optical stack is switchablebetween at least two modes.

Another embodiment of the present disclosure sets forth a method. Themethod includes detecting a pupil position of an eye of a user. Themethod further includes determining an angle to steer light based on thedetected pupil position. In addition, the method includes steering thelight at the angle using at least an optical stack comprising aplurality of Pancharatnam-Berry Phase (PBP) gratings, where the opticalstack is switchable between at least two modes.

One advantage of the foveated display systems disclosed herein is thatthe foveated display systems generate high-resolution virtual imageryfor a foveal region of a user's eye gaze along with low-resolution,large field of view background imagery for other regions of the user'seye gaze. A diffuser that is used to generate the projected imagery canbe disposed within a pancake lens, which is a relatively compact (i.e.,thinner) design that is beneficial for applications with a HMD or otherdevices where a small form factor and weight are considerations. Inaddition, a switchable Pancharatnam-Berry phase grating stack can beused to increase the steering range of a beam-steering device used togenerate the high-resolution virtual imagery such that, e.g., lightassociated with the virtual imagery can be steered to cover an entirefield of view that is visible to the user's eye. These technicaladvantages represent one or more technological advancements over priorart approaches.

BRIEF DESCRIPTION OF THE DRAWINGS

So that the manner in which the above recited features of the variousembodiments can be understood in detail, a more particular descriptionof the disclosed concepts, briefly summarized above, may be had byreference to various embodiments, some of which are illustrated in theappended drawings. It is to be noted, however, that the appendeddrawings illustrate only typical embodiments of the disclosed conceptsand are therefore not to be considered limiting of scope in any way, andthat there are other equally effective embodiments.

FIG. 1A is a diagram of a near eye display (NED), according to variousembodiments.

FIG. 1B is a cross section of the front rigid body of the embodiments ofthe NED illustrated in FIG. 1A.

FIG. 2A is a diagram of a head-mounted display (HMD) implemented as aNED, according to various embodiments.

FIG. 2B is a cross-section view of the HMD of FIG. 2A implemented as anear eye display, according to various embodiments.

FIG. 3 is a block diagram of a NED system, according to variousembodiments.

FIG. 4 is a schematic diagram illustrating a foveated display system,according to various embodiments.

FIG. 5 illustrates in greater detail components of a foveated displaysystem, according to various embodiments.

FIG. 6 is a schematic diagram illustrating a pancake lens that includesa diffuser, according to various embodiments.

FIG. 7 is a ray-tracing diagram illustrating operation of a pancake lensthat includes a diffuser, according to various embodiments.

FIG. 8 is a schematic diagram illustrating an optical configuration ofthe foveal display module of FIG. 4 , according to various embodiments.

FIG. 9 illustrates in greater detail components of the foveal displaymodule of FIG. 4 , according to various embodiments.

FIG. 10 illustrates components and operation of a switchablePancharatnam-Berry phase (PBP) grating stack, according to variousembodiments.

FIG. 11 illustrates a PBP grating, according to various embodiments.

FIG. 12 illustrates a method for generating artificial reality contentusing a foveated display system, according to various embodiments.

FIG. 13 illustrates in greater detail one of the steps of the method ofFIG. 12 , according to various embodiments.

FIG. 14 illustrates in greater detail another of the steps of the methodof FIG. 12 , according to various embodiments.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth toprovide a more thorough understanding of the various embodiments.However, it is apparent to one of skilled in the art that the disclosedconcepts may be practiced without one or more of these specific details.

Configuration Overview

One or more embodiments disclosed herein relate to a foveated displaysystem. The foveated display system includes a peripheral display moduledisposed in series with a foveal display module. The peripheral displaymodule is configured to generate low-resolution, large field of view(FOV) imagery for a user's peripheral vision, while the foveal displaymodule is configured to perform foveated rendering in whichhigh-resolution imagery is focused towards a foveal region of the user'seye gaze. In addition, real-world light can pass through the peripheraland foveal display modules and be observed by the user.

The peripheral display module includes a projection device that projectsbackground imagery for a user's peripheral vision onto a diffuser thatdiffuses the background imagery, as well as a pancake lens thatincreases the propagating distance of light such that the backgroundimagery appears further away to the user. The diffuser is polarization,angular, and wavelength selective in some embodiments. Such a diffusermay be constructed using, e.g., a cholesteric liquid crystal material.In operation, circularly polarized light is projected onto the diffuserat a slanted angle and bounces twice within the pancake lens. In someembodiments, the diffuser may also be included within the pancake lens,which is a more compact (i.e., thinner) design than one in which thediffuser is external to the pancake lens.

The foveal display module includes a holographic display, abeam-steering device such as a micro-electro-mechanical system (MEMS)mirror, an angular- and wavelength-selective lens such as a holographicoptical element (HOE) lens, and an eye tracking device. In operation,the beam-steering device is controllable to focus light from theholographic display towards a foveal region of a user's eye gaze via theangular- and wavelength-selective lens, based on a pupil positioncaptured by the eye tracking device. In some embodiments, the fovealdisplay module may also include a switchable Pancharatnam-Berry Phase(PBP) grating stack that increases a steering range of the beam-steeringdevice. In such cases, the switchable PBP grating stack may include aswitchable half-wave plate disposed between two PBP gratings. Thediffraction angle produced by one PBP grating in the switchable PBPgrating stack differs based on a handedness of polarization of lightoutput by the switchable half-wave plate when the switchable half-waveplate is on versus when the switchable half-wave plate is off.

Embodiments of the disclosure may also include or be implemented inconjunction with an artificial reality system. Artificial reality is aform of reality that has been adjusted in some manner beforepresentation to a user, which may include, for example, a virtualreality (VR) system, an augmented reality (AR) system, a mixed reality(MR) system, a hybrid reality system, or some combination and/orderivatives thereof. Artificial reality content may include, withoutlimitation, completely generated content or generated content combinedwith captured (e.g., real-world) content. The artificial reality contentmay include, without limitation, video, audio, haptic feedback, or somecombination thereof. The artificial reality content may be presented ina single channel or in multiple channels (such as stereo video thatproduces a three-dimensional effect to the viewer). Additionally, insome embodiments, artificial reality systems may also be associated withapplications, products, accessories, services, or some combinationthereof, that are used to, e.g., create content in an artificial realitysystem and/or are otherwise used in (e.g., perform activities in) anartificial reality system. The artificial reality system may beimplemented on various platforms, including a head-mounted display (HMD)connected to a host computer system, a standalone HMD, a mobile deviceor computing system, or any other hardware platform capable of providingartificial reality content to one or more viewers.

Embodiments of the disclosure may also include or be implemented inconjunction with an artificial reality system. Artificial reality is aform of reality that has been adjusted in some manner beforepresentation to a user, which may include, for example, a VR system, anAR system, a MR system, a hybrid reality system, or some combinationand/or derivatives thereof. Artificial reality content may include,without limitation, completely generated content or generated contentcombined with captured (e.g., real-world) content. The artificialreality content may include, without limitation, video, audio, hapticfeedback, or some combination thereof. The artificial reality contentmay be presented in a single channel or in multiple channels (such asstereo video that produces a three-dimensional effect to the viewer).Additionally, in some embodiments, artificial reality systems may alsobe associated with applications, products, accessories, services, orsome combination thereof, that are used to, e.g., create content in anartificial reality system and/or are otherwise used in (e.g., performactivities in) an artificial reality system. The artificial realitysystem may be implemented on various platforms, including a HMDconnected to a host computer system, a standalone HMD, a mobile deviceor computing system, or any other hardware platform capable of providingartificial reality content to one or more viewers.

System Overview

FIG. 1A is a wire diagram of a near eye display (NED) 100, according tovarious embodiments. Although NEDs and head mounted displays (HMDs) aredisclosed herein as reference examples, display devices that includefoveated display systems may also be configured for placement inproximity of an eye or eyes of a user at a fixed location, without beinghead-mounted (e.g., the display device may be mounted in a vehicle, suchas a car or an airplane, for placement in front of an eye or eyes of theuser).

As shown, the NED 100 includes a front rigid body 105 and a band 110.The front rigid body 105 includes one or more electronic displayelements of an electronic display (not shown), an inertial measurementunit (IMU) 115, one or more position sensors 120, and locators 125. Asillustrated in FIG. 1A, position sensors 120 are located within the IMU115, and neither the IMU 115 nor the position sensors 120 are visible tothe user. In various embodiments, where the NED 100 acts as an AR or MRdevice, portions of the NED 100 and/or its internal components are atleast partially transparent.

FIG. 1B is a cross section 160 of the front rigid body 105 of theembodiments of the NED 100 illustrated in FIG. 1A. As shown, the frontrigid body 105 includes an electronic display 130 and an optics block135 that together provide image light to an exit pupil 145. The exitpupil 145 is the location of the front rigid body 105 where a user's eye140 may be positioned. For purposes of illustration, FIG. 1B shows across section 160 associated with a single eye 140, but another opticsblock, separate from the optics block 135, may provide altered imagelight to another eye of the user. Additionally, the NED 100 includes aneye tracking system (not shown in FIG. 1B). The eye tracking system mayinclude one or more sources that illuminate one or both eyes of theuser. The eye tracking system may also include one or more cameras thatcapture images of one or both eyes of the user to track the positions ofthe eyes.

The electronic display 130 displays images to the user. In variousembodiments, the electronic display 130 may comprise a single electronicdisplay or multiple electronic displays (e.g., a display for each eye ofa user). Examples of the electronic display 130 include: a liquidcrystal display (LCD), an organic light emitting diode (OLED) display,an active-matrix organic light-emitting diode display (AMOLED), a QOLED,a QLED, some other display, or some combination thereof.

The optics block 135 adjusts an orientation of image light emitted fromthe electronic display 130 such that the electronic display 130 appearsat particular virtual image distances from the user. The optics block135 is configured to receive image light emitted from the electronicdisplay 130 and direct the image light to an eye-box associated with theexit pupil 145. The image light directed to the eye-box forms an imageat a retina of eye 140. The eye-box is a region defining how much theeye 140 moves up/down/left/right from without significant degradation inthe image quality. In the illustration of FIG. 1B, a field of view (FOV)150 is the extent of the observable world that is seen by the eye 140 atany given moment.

Additionally, in some embodiments, the optics block 135 magnifiesreceived light, corrects optical errors associated with the image light,and presents the corrected image light to the eye 140. The optics block135 may include one or more optical elements 155 in optical series. Anoptical element 155 may be an aperture, a Fresnel lens, a convex lens, aconcave lens, a filter, a waveguide, a Pancharatnam-Berry phase (PBP)lens or grating, a color-selective filter, a waveplate, a C-plate, orany other suitable optical element 155 that affects the image light.Moreover, the optics block 135 may include combinations of differentoptical elements. One or more of the optical elements in the opticsblock 135 may have one or more coatings, such as anti-reflectivecoatings. In some embodiments, the optics block 135 may include opticalelements of one or more of the foveated, peripheral, and/or fovealsystems discussed in detail below in conjunction with FIGS. 4-11 .

FIG. 2A is a diagram of an HMD 162 implemented as a NED, according tovarious embodiments. As shown, the HMD 162 is in the form of a pair ofaugmented reality glasses. The HMD 162 presents computer-generated mediato a user and augments views of a physical, real-world environment withthe computer-generated media. Examples of computer-generated mediapresented by the HMD 162 include one or more images, video, audio, orsome combination thereof. In some embodiments, audio is presented via anexternal device (e.g., speakers and headphones) that receives audioinformation from the HMD 162, a console (not shown), or both, andpresents audio data based on audio information. In some embodiments, theHMD 162 may be modified to also operate as a VR HMD, a MR HMD, or somecombination thereof. The HMD 162 includes a frame 175 and a display 164.As shown, the frame 175 mounts the NED to the user's head, while thedisplay 164 provides image light to the user. The display 164 may becustomized to a variety of shapes and sizes to conform to differentstyles of eyeglass frames.

FIG. 2B is a cross-section view of the HMD 162 of FIG. 2A implemented asa NED, according to various embodiments. This view includes frame 175,display 164 (which comprises a display assembly 180 and a display block185), and eye 170. The display assembly 180 supplies image light to theeye 170. The display assembly 180 houses display block 185, which, indifferent embodiments, encloses the different types of imaging opticsand redirection structures. For purposes of illustration, FIG. 2B showsthe cross section associated with a single display block 185 and asingle eye 170, but in alternative embodiments not shown, anotherdisplay block, which is separate from display block 185 shown in FIG.2B, provides image light to another eye of the user.

The display block 185, as illustrated, is configured to combine lightfrom a local area with light from computer generated image to form anaugmented scene. The display block 185 is also configured to provide theaugmented scene to the eyebox 165 corresponding to a location of theuser's eye 170. The display block 185 may include, for example, awaveguide display, a focusing assembly, a compensation assembly, or somecombination thereof. In some embodiments, the display block 185 mayinclude one or more components of the foveated, peripheral, and/orfoveal systems discussed in detail below in conjunction with FIGS. 4-11.

HMD 162 may include one or more other optical elements between thedisplay block 185 and the eye 170. The optical elements may act to, forexample, correct aberrations in image light emitted from the displayblock 185, magnify image light emitted from the display block 185, someother optical adjustment of image light emitted from the display block185, or some combination thereof. The example for optical elements mayinclude an aperture, a Fresnel lens, a convex lens, a concave lens, afilter, or any other suitable optical element that affects image light.The display block 185 may also comprise one or more materials (e.g.,plastic, glass, etc.) with one or more refractive indices thateffectively minimize the weight and widen a field of view of the HMD162.

FIG. 3 is a block diagram of an embodiment of a near eye display system300 in which a console 310 operates. In some embodiments, the NED system300 corresponds to the NED 100 or the HMD 162. The NED system 300 mayoperate in a VR system environment, an AR system environment, a MRsystem environment, or some combination thereof. The NED system 300shown in FIG. 3 comprises a NED 305 and an input/output (I/O) interface315 that is coupled to the console 310.

While FIG. 3 shows an example NED system 300 including one NED 305 andone I/O interface 315, in other embodiments any number of thesecomponents may be included in the NED system 300. For example, there maybe multiple NEDs 305 that each has an associated I/O interface 315,where each NED 305 and I/O interface 315 communicates with the console310. In alternative configurations, different and/or additionalcomponents may be included in the NED system 300. Additionally, variouscomponents included within the NED 305, the console 310, and the I/Ointerface 315 may be distributed in a different manner than is describedin conjunction with FIG. 3 in some embodiments. For example, some or allof the functionality of the console 310 may be provided by the NED 305.

The NED 305 may be a head-mounted display that presents content to auser. The content may include virtual and/or augmented views of aphysical, real-world environment including computer-generated elements(e.g., two-dimensional or three-dimensional images, two-dimensional orthree-dimensional video, sound, etc.). In some embodiments, the NED 305may also present audio content to a user. The NED 305 and/or the console310 may transmit the audio content to an external device via the I/Ointerface 315. The external device may include various forms of speakersystems and/or headphones. In various embodiments, the audio content issynchronized with visual content being displayed by the NED 305.

The NED 305 may comprise one or more rigid bodies, which may be rigidlyor non-rigidly coupled together. A rigid coupling between rigid bodiescauses the coupled rigid bodies to act as a single rigid entity. Incontrast, a non-rigid coupling between rigid bodies allows the rigidbodies to move relative to each other.

As shown in FIG. 3 , the NED 305 may include a depth camera assembly(DCA) 320, a display 325, an optical assembly 330, one or more positionsensors 335, an inertial measurement unit (IMU) 340, an eye trackingsystem 345, and a varifocal module 350. In some embodiments, the display325 and the optical assembly 330 can be integrated together into aprojection assembly. Various embodiments of the NED 305 may haveadditional, fewer, or different components than those listed above.Additionally, the functionality of each component may be partially orcompletely encompassed by the functionality of one or more othercomponents in various embodiments.

The DCA 320 captures sensor data describing depth information of an areasurrounding the NED 305. The sensor data may be generated by one or acombination of depth imaging techniques, such as triangulation,structured light imaging, time-of-flight imaging, laser scan, and soforth. The DCA 320 can compute various depth properties of the areasurrounding the NED 305 using the sensor data. Additionally oralternatively, the DCA 320 may transmit the sensor data to the console310 for processing.

The DCA 320 includes an illumination source, an imaging device, and acontroller. The illumination source emits light onto an area surroundingthe NED 305. In an embodiment, the emitted light is structured light.The illumination source includes a plurality of emitters that each emitslight having certain characteristics (e.g., wavelength, polarization,coherence, temporal behavior, etc.). The characteristics may be the sameor different between emitters, and the emitters can be operatedsimultaneously or individually. In one embodiment, the plurality ofemitters could be, e.g., laser diodes (such as edge emitters), inorganicor organic light-emitting diodes (LEDs), a vertical-cavitysurface-emitting laser (VCSEL), or some other source. In someembodiments, a single emitter or a plurality of emitters in theillumination source can emit light having a structured light pattern.The imaging device captures ambient light in the environment surroundingNED 305, in addition to light reflected off of objects in theenvironment that is generated by the plurality of emitters. In variousembodiments, the imaging device may be an infrared camera or a cameraconfigured to operate in a visible spectrum. The controller coordinateshow the illumination source emits light and how the imaging devicecaptures light. For example, the controller may determine a brightnessof the emitted light. In some embodiments, the controller also analyzesdetected light to detect objects in the environment and positioninformation related to those objects.

The display 325 displays two-dimensional or three-dimensional images tothe user in accordance with pixel data received from the console 310. Invarious embodiments, the display 325 comprises a single display ormultiple displays (e.g., separate displays for each eye of a user). Insome embodiments, the display 325 comprises a single or multiplewaveguide displays. Light can be coupled into the single or multiplewaveguide displays via, e.g., a liquid crystal display (LCD), an organiclight emitting diode (OLED) display, an inorganic light emitting diode(ILED) display, an active-matrix organic light-emitting diode (AMOLED)display, a transparent organic light emitting diode (TOLED) display, alaser-based display, one or more waveguides, other types of displays, ascanner, a one-dimensional array, and so forth. In addition,combinations of the displays types may be incorporated in display 325and used separately, in parallel, and/or in combination.

The optical assembly 330 magnifies image light received from the display325, corrects optical errors associated with the image light, andpresents the corrected image light to a user of the NED 305. The opticalassembly 330 includes a plurality of optical elements. For example, oneor more of the following optical elements may be included in the opticalassembly 330: an aperture, a Fresnel lens, a convex lens, a concavelens, a filter, a reflecting surface, or any other suitable opticalelement that deflects, reflects, refracts, and/or in some way altersimage light. Moreover, the optical assembly 330 may include combinationsof different optical elements. In some embodiments, one or more of theoptical elements in the optical assembly 330 may have one or morecoatings, such as partially reflective or antireflective coatings. Theoptical assembly 330 can be integrated into a projection assembly, e.g.,a projection assembly. In one embodiment, the optical assembly 330includes the optics block 155.

In operation, the optical assembly 330 magnifies and focuses image lightgenerated by the display 325. In so doing, the optical assembly 330enables the display 325 to be physically smaller, weigh less, andconsume less power than displays that do not use the optical assembly330. Additionally, magnification may increase the field of view of thecontent presented by the display 325. For example, in some embodiments,the field of view of the displayed content partially or completely usesa user's field of view. For example, the field of view of a displayedimage may meet or exceed 310 degrees. In various embodiments, the amountof magnification may be adjusted by adding or removing optical elements.

In some embodiments, the optical assembly 330 may be designed to correctone or more types of optical errors. Examples of optical errors includebarrel or pincushion distortions, longitudinal chromatic aberrations, ortransverse chromatic aberrations. Other types of optical errors mayfurther include spherical aberrations, chromatic aberrations or errorsdue to the lens field curvature, astigmatisms, in addition to othertypes of optical errors. In some embodiments, visual content transmittedto the display 325 is pre-distorted, and the optical assembly 330corrects the distortion as image light from the display 325 passesthrough various optical elements of the optical assembly 330. In someembodiments, optical elements of the optical assembly 330 are integratedinto the display 325 as a projection assembly that includes at least onewaveguide coupled with one or more optical elements. In some embodimentsthe display 325 and/or the optical assembly 330 may include theperipheral display systems or components thereof discussed below inconjunction with FIGS. 4-7 .

The IMU 340 is an electronic device that generates data indicating aposition of the NED 305 based on measurement signals received from oneor more of the position sensors 335 and from depth information receivedfrom the DCA 320. In some embodiments of the NED 305, the IMU 340 may bea dedicated hardware component. In other embodiments, the IMU 340 may bea software component implemented in one or more processors.

In operation, a position sensor 335 generates one or more measurementsignals in response to a motion of the NED 305. Examples of positionsensors 335 include: one or more accelerometers, one or more gyroscopes,one or more magnetometers, one or more altimeters, one or moreinclinometers, and/or various types of sensors for motion detection,drift detection, and/or error detection. The position sensors 335 may belocated external to the IMU 340, internal to the IMU 340, or somecombination thereof.

Based on the one or more measurement signals from one or more positionsensors 335, the IMU 340 generates data indicating an estimated currentposition of the NED 305 relative to an initial position of the NED 305.For example, the position sensors 335 may include multipleaccelerometers to measure translational motion (forward/back, up/down,left/right) and multiple gyroscopes to measure rotational motion (e.g.,pitch, yaw, and roll). In some embodiments, the IMU 340 rapidly samplesthe measurement signals and calculates the estimated current position ofthe NED 305 from the sampled data. For example, the IMU 340 mayintegrate the measurement signals received from the accelerometers overtime to estimate a velocity vector and integrates the velocity vectorover time to determine an estimated current position of a referencepoint on the NED 305. Alternatively, the IMU 340 provides the sampledmeasurement signals to the console 310, which analyzes the sample datato determine one or more measurement errors. The console 310 may furthertransmit one or more of control signals and/or measurement errors to theIMU 340 to configure the IMU 340 to correct and/or reduce one or moremeasurement errors (e.g., drift errors). The reference point is a pointthat may be used to describe the position of the NED 305. The referencepoint may generally be defined as a point in space or a position relatedto a position and/or orientation of the NED 305.

In various embodiments, the IMU 340 receives one or more parameters fromthe console 310. The one or more parameters are used to maintaintracking of the NED 305. Based on a received parameter, the IMU 340 mayadjust one or more IMU parameters (e.g., a sample rate). In someembodiments, certain parameters cause the IMU 340 to update an initialposition of the reference point so that it corresponds to a nextposition of the reference point. Updating the initial position of thereference point as the next calibrated position of the reference pointhelps reduce drift errors in detecting a current position estimate ofthe IMU 340.

In some embodiments, the eye tracking system 345 is integrated into theNED 305. The eye-tracking system 345 may comprise one or moreillumination sources and an imaging device (camera). In operation, theeye tracking system 345 generates and analyzes tracking data related toa user's eyes as the user wears the NED 305. The eye tracking system 345may further generate eye tracking information that may compriseinformation about a position of the user's eye, i.e., information aboutan angle of an eye-gaze.

In some embodiments, the varifocal module 350 is further integrated intothe NED 305. The varifocal module 350 may be communicatively coupled tothe eye tracking system 345 in order to enable the varifocal module 350to receive eye tracking information from the eye tracking system 345.The varifocal module 350 may further modify the focus of image lightemitted from the display 325 based on the eye tracking informationreceived from the eye tracking system 345. Accordingly, the varifocalmodule 350 can reduce vergence-accommodation conflict that may beproduced as the user's eyes resolve the image light. In variousembodiments, the varifocal module 350 can be interfaced (e.g., eithermechanically or electrically) with at least one optical element of theoptical assembly 330.

In operation, the varifocal module 350 may adjust the position and/ororientation of one or more optical elements in the optical assembly 330in order to adjust the focus of image light propagating through theoptical assembly 330. In various embodiments, the varifocal module 350may use eye tracking information obtained from the eye tracking system345 to determine how to adjust one or more optical elements in theoptical assembly 330. In some embodiments, the varifocal module 350 mayperform foveated rendering of the image light based on the eye trackinginformation obtained from the eye tracking system 345 in order to adjustthe resolution of the image light emitted by the display 325. In thiscase, the varifocal module 350 configures the display 325 to display ahigh pixel density in a foveal region of the user's eye-gaze and a lowpixel density in other regions of the user's eye-gaze. In someembodiments, the varifocal module 350 may include the foveal displaysystems or components thereof that are discussed below in conjunctionwith FIGS. 4-5 and 8-11 .

The I/O interface 315 facilitates the transfer of action requests from auser to the console 310. In addition, the I/O interface 315 facilitatesthe transfer of device feedback from the console 310 to the user. Anaction request is a request to perform a particular action. For example,an action request may be an instruction to start or end capture of imageor video data or an instruction to perform a particular action within anapplication, such as pausing video playback, increasing or decreasingthe volume of audio playback, and so forth. In various embodiments, theI/O interface 315 may include one or more input devices. Example inputdevices include: a keyboard, a mouse, a game controller, a joystick,and/or any other suitable device for receiving action requests andcommunicating the action requests to the console 310. In someembodiments, the I/O interface 315 includes an IMU 340 that capturescalibration data indicating an estimated current position of the I/Ointerface 315 relative to an initial position of the I/O interface 315.

In operation, the I/O interface 315 receives action requests from theuser and transmits those action requests to the console 310. Responsiveto receiving the action request, the console 310 performs acorresponding action. For example, responsive to receiving an actionrequest, the console 310 may configure the I/O interface 315 to emithaptic feedback onto an arm of the user. For example, the console 315may configure the I/O interface 315 to deliver haptic feedback to a userwhen an action request is received. Additionally or alternatively, theconsole 310 may configure the I/O interface 315 to generate hapticfeedback when the console 310 performs an action, responsive toreceiving an action request.

The console 310 provides content to the NED 305 for processing inaccordance with information received from one or more of: the DCA 320,the NED 305, and the I/O interface 315. As shown in FIG. 3 , the console310 includes an application store 355, a tracking module 360, and anengine 365. In some embodiments, the console 310 may have additional,fewer, or different modules and/or components than those described inconjunction with FIG. 3 . Similarly, the functions further describedbelow may be distributed among components of the console 310 in adifferent manner than described in conjunction with FIG. 3 .

The application store 355 stores one or more applications for executionby the console 310. An application is a group of instructions that, whenexecuted by a processor, performs a particular set of functions, such asgenerating content for presentation to the user. For example, anapplication may generate content in response to receiving inputs from auser (e.g., via movement of the NED 305 as the user moves his/her head,via the I/O interface 315, etc.). Examples of applications include:gaming applications, conferencing applications, video playbackapplications, or other suitable applications.

The tracking module 360 calibrates the NED system 300 using one or morecalibration parameters. The tracking module 360 may further adjust oneor more calibration parameters to reduce error in determining a positionand/or orientation of the NED 305 or the I/O interface 315. For example,the tracking module 360 may transmit a calibration parameter to the DCA320 in order to adjust the focus of the DCA 320. Accordingly, the DCA320 may more accurately determine positions of structured light elementsreflecting off of objects in the environment. The tracking module 360may also analyze sensor data generated by the IMU 340 in determiningvarious calibration parameters to modify. Further, in some embodiments,if the NED 305 loses tracking of the user's eye, then the trackingmodule 360 may re-calibrate some or all of the components in the NEDsystem 300. For example, if the DCA 320 loses line of sight of at leasta threshold number of structured light elements projected onto theuser's eye, the tracking module 360 may transmit calibration parametersto the varifocal module 350 in order to re-establish eye tracking.

The tracking module 360 tracks the movements of the NED 305 and/or ofthe I/O interface 315 using information from the DCA 320, the one ormore position sensors 335, the IMU 340 or some combination thereof. Forexample, the tracking module 360 may determine a reference position ofthe NED 305 from a mapping of an area local to the NED 305. The trackingmodule 360 may generate this mapping based on information received fromthe NED 305 itself. The tracking module 360 may also utilize sensor datafrom the IMU 340 and/or depth data from the DCA 320 to determinereferences positions for the NED 305 and/or I/O interface 315. Invarious embodiments, the tracking module 360 generates an estimationand/or prediction for a subsequent position of the NED 305 and/or theI/O interface 315. The tracking module 360 may transmit the predictedsubsequent position to the engine 365.

The engine 365 generates a three-dimensional mapping of the areasurrounding the NED 305 (i.e., the “local area”) based on informationreceived from the NED 305. In some embodiments, the engine 365determines depth information for the three-dimensional mapping of thelocal area based on depth data received from the DCA 320 (e.g., depthinformation of objects in the local area). In some embodiments, theengine 365 calculates a depth and/or position of the NED 305 by usingdepth data generated by the DCA 320. In particular, the engine 365 mayimplement various techniques for calculating the depth and/or positionof the NED 305, such as stereo based techniques, structured lightillumination techniques, time-of-flight techniques, and so forth. Invarious embodiments, the engine 365 uses depth data received from theDCA 320 to update a model of the local area and to generate and/ormodify media content based in part on the updated model.

The engine 365 also executes applications within the NED system 300 andreceives position information, acceleration information, velocityinformation, predicted future positions, or some combination thereof, ofthe NED 305 from the tracking module 360. Based on the receivedinformation, the engine 365 determines various forms of media content totransmit to the NED 305 for presentation to the user. For example, ifthe received information indicates that the user has looked to the left,the engine 365 generates media content for the NED 305 that mirrors theuser's movement in a virtual environment or in an environment augmentingthe local area with additional media content. Accordingly, the engine365 may generate and/or modify media content (e.g., visual and/or audiocontent) for presentation to the user. The engine 365 may furthertransmit the media content to the NED 305. Additionally, in response toreceiving an action request from the I/O interface 315, the engine 365may perform an action within an application executing on the console310. The engine 305 may further provide feedback when the action isperformed. For example, the engine 365 may configure the NED 305 togenerate visual and/or audio feedback and/or the I/O interface 315 togenerate haptic feedback to the user.

In some embodiments, based on the eye tracking information (e.g.,orientation of the user's eye) received from the eye tracking system345, the engine 365 determines a resolution of the media contentprovided to the NED 305 for presentation to the user on the display 325.The engine 365 may adjust a resolution of the visual content provided tothe NED 305 by configuring the display 325 to perform foveated renderingof the visual content, based at least in part on a direction of theuser's gaze received from the eye tracking system 345. The engine 365provides the content to the NED 305 having a high resolution on thedisplay 325 in a foveal region of the user's gaze and a low resolutionin other regions, thereby reducing the power consumption of the NED 305.In addition, using foveated rendering reduces a number of computingcycles used in rendering visual content without compromising the qualityof the user's visual experience. In some embodiments, the engine 365 canfurther use the eye tracking information to adjust a focus of the imagelight emitted from the display 325 in order to reducevergence-accommodation conflicts. In some embodiments, the engine 365may interoperate with one or more of the foveated, peripheral, and/orfoveal systems, or components thereof, that are discussed in detailbelow in conjunction with FIGS. 4-11 .

Foveated Display System

FIG. 4 is a schematic diagram illustrating a foveated display system400, according to various embodiments. As shown, the foveated displaysystem 400 includes a peripheral display module 402 and a foveal displaymodule 410. For purposes of illustration, FIGS. 4-11 show a singlefoveated display system and components thereof that provide image lightto one eye of a user. In some embodiments not shown, another separatefoveated display system may provide image light to another eye of theuser.

In operation, the foveated display system 400 is configured to generatehigh-resolution virtual imagery via foveated rendering for a fovealregion of a user's eye gaze, as well as low-resolution, large field ofview (FOV) background imagery for other regions of the user's eye gaze.In particular, the foveal display module 410 is configured to generatethe high-resolution virtual imagery, while the peripheral display module402 is configured to generate the low-resolution, large FOV backgroundimagery. In addition, the peripheral and foveal display modules 402 and410 are configured to permit real-world light to pass through and beobserved by the user.

As shown, the peripheral display module 402 includes a projection device404, a diffuser 406, and a pancake lens 408. The foveal display module410 includes a holographic display 412, a beam-steering device 414, anangular- and wavelength-selective lens 416, and an eye-tracking module418. Illustratively, the diffuser 406 and the angular- andwavelength-selective lens 416 are in-line with one another. That is, thediffuser 406 and the angular- and wavelength-selective lens 416 share acommon axis.

In operation, the projection device 404 emits polarized lightcorresponding to generated imagery. As shown, the polarized light isprojected at a slanted angle onto the diffuser 406, which reflects anddiffuses such light due to polarization and angular selectivitycharacteristics of the diffuser 406. In some embodiments, the diffuser406 may be polarization, angular, and wavelength selective. In suchcases, the diffuser 406 may permit most light to pass through, butdiffuse light having a particular handedness of polarization that iswithin a particular range of wavelengths and incident on the diffuser406 within a particular range of angles. More generally, anytechnically-feasible diffuser may be used that is able to diffuse lightfrom the projection device 404 that is projected thereon whilepermitting other light (e.g., real-world light) to pass through.

Light diffused by the diffuser 406 provides low-resolution, high FOVbackground imagery for the non-foveal regions of a user's eye gaze. Inaddition, real-world light, i.e., light from a real-world scene, that isincident on the diffuser 406 passes through the diffuser 406 withoutbeing diffused due to the polarization, angular, and wavelengthselectivity characteristics of the diffuser 406. As a result, the usercan observe both the low-resolution, high FOV background imagerygenerated using the diffuser 406 and real-world content.

As shown, light diffused by the diffuser 406 is passed through thepancake lens 408. A pancake lens is a folded optic in which light thatenters reflects, or “bounces,” through multiple times before exiting.The multiple bounces increase the propagation distance of light, whichcan in turn increase the perceived distance of imagery from a userand/or magnify the imagery. By increase the propagating distance oflight, the pancake lens 408 causes the low-resolution, high FOVbackground imagery generated via the diffuser 406 to appear further awayfrom a user. Illustratively, the pancake lens 408 also has focal power,which can make the low-resolution, high FOV background imagery appeareven further away. Although a pancake lens is described herein as areference example, in other embodiments, any technically-feasibleoptical element(s) may be used to increase the propagating distance oflight or otherwise increase the perceived distance of imagery from auser and/or magnify the imagery.

Although shown as distinct components for illustrative purposes, in someembodiments the diffuser 406 may be included within the pancake lens408. Such embodiments are discussed in greater detail below inconjunction with FIGS. 6-7 . That is, the diffuser 406 may generally beincluded within, or be external to, the pancake lens 408. Configurationsin which the diffuser 406 is included within the pancake lens 408 aremore compact than configurations in which the diffuser 406 is distinctfrom the pancake lens 408. Such compactness can be beneficial forapplications with a HMD or other devices where a small form factor andweight are considerations.

As shown, light that has passed through the pancake lens 408 furtherpasses through the angular- and wavelength-selective lens 416 of thefoveal display module 410 toward an eye box. In some embodiments, theangular- and wavelength-selective lens 416 also has focal power. Inoperation, such an angular- and wavelength-selective lens 416 may allowthrough most light, including the light that has passed through thepancake lens 408, while reflecting and focusing light that is within aparticular range of wavelengths and incident on the lens 416 within aparticular range of angles, including light from the holographic display412 that is steered onto the angular- and wavelength-selective lens 416by the beam-steering device 414.

As shown, the beam-steering device 414 is a beam-steering mirror. Thebeam-steering mirror 414 is configured to perform gaze-followingsteering in which the beam-steering mirror 414 steers light from theholographic display 412 toward a foveal region of a user's eye gaze viathe angular- and wavelength-selective lens 416, thereby producinghigh-resolution virtual imagery that can be observed by the user. Insome embodiments, the beam-steering mirror 414 may be amicroelectro-mechanical system (MEMS) mirror. Although such a MEMSmirror is described herein as a reference example, in other embodiments,any technically-feasible device may be used to steer light toward afoveal region of a user's eye gaze.

As shown, the angular- and wavelength-selective lens 416 reflects lightfrom the holographic display 412 that is focused onto the lens 416 atvarious angles by the beam-steering mirror 414, due to the wavelengthand angular selectivity characteristics of the lens 416. In someembodiments, the angular- and wavelength-selective lens 416 may be aholographic optical element (HOE), such as a volume grating lens. A HOEis an optical element produced using holographic imaging processes orprinciples. Although discussed herein primarily with respect to a HOEfor illustrative purposes, any optical element(s) that performfunctionalities of the angular- and wavelength-selective lens 416described herein may be used in other embodiments.

The holographic display 412 is a display that uses light diffraction tocreate a virtual image. In some embodiments, the holographic display 412may include a spatial light modulator that is configured to modulatelight emitted by a projection device. Further, light produced by theholographic display 412 may be within a wavelength range that isreflected by the angular- and wavelength-selective lens 416 whenincident thereon within a particular range of angles. Although discussedherein primarily with respect to a holographic display for illustrativepurposes, in other embodiments, any technically-feasible displaydevice(s) capable of generating light that can be focused on a fovealregion of a user's eye gaze to produce high-resolution imagery may beused.

As the foregoing illustrates, a user of the foveated display system 400can observe AR content that includes (1) high-resolution virtual imageryfocused on a foveal region of the user's eye gaze by the foveal displaymodule 410, (2) low resolution, high FOV background imagery produced bythe peripheral display module 402, and (3) content from a real-worldscene. Although AR is described herein as a reference example, it shouldbe understood that the foveated display system 400 may also be used inother artificial reality applications, such as VR, MR, hybrid reality,or some combination and/or derivative thereof. For example, in the caseof VR, real-world light would not be included in the output of thefoveated display system 400.

FIG. 5 illustrates in greater detail components of a foveated displaysystem 500, according to various embodiments. As shown, the foveateddisplay system 500 includes a peripheral display module 502 and a fovealdisplay module 512, which correspond to the peripheral display module402 and the foveal display module 410, respectively, that are describedabove in conjunction with FIG. 4 . In operation, the foveal displaymodule 512 is configured to generate high-resolution virtual imagery viafoveated rendering for a foveal region of a user's eye gaze, while theperipheral display module 502 is configured to generate low-resolution,large FOV background imagery for other regions of the user's eye gaze.

As shown, the peripheral display module 502 includes a projection device502, which may comprise any technically-feasible light source, such as alight-emitting diode (LED) device, an organic LED (OLED), a laser, etc.In some embodiments, the projection device 502 may be a pico projector.

The peripheral display module 502 further includes a condenser 506 thatis configured to render a divergent light beam from the projectiondevice 502 into a parallel beam that is projected onto a diffuser 508.The diffuser 508 is a polarization, angular, and wavelength selectivediffuser, similar to the diffuser 406 described above in conjunctionwith FIG. 4 . A diffuser having characteristics of the diffuser 508described herein, including polarization, angular, and wavelengthselectivity, may be constructed using cholesteric liquid crystalmaterials. In some embodiments, the diffuser 508 may be a volume gratinglens constructed using a cholesteric liquid crystal material. Moregenerally, any technically feasible material(s) may be used to constructthe diffuser 508.

Similar to the discussion above in conjunction with FIG. 4 , theprojection device 502 emits polarized light associated with generatedimagery that is projected, via the condenser 506, onto the diffuser 508at a slanted angle. Due to the polarization and angular selectivitycharacteristics of the diffuser 508, the diffuser 508 is configured toreflect and diffuse the projected light toward an eye box. Suchdiffusion produces low-resolution, high FOV background imagery fornon-foveal regions of a user's eye gaze.

As described, the polarization and angular selectivity characteristicsof the diffuser 508 cause the diffuser 508 to only diffuse or scatterlight having a particular handedness of polarization that is incident onthe diffuser 508 within a particular range of angles. For example, in aparticular embodiment, the diffuser 508 could selectively diffuse lightthat has one handedness of polarization and is incident on the diffuser508 at an angle into 40°, while permitting other light to pass throughwithout being diffused. In some embodiments, the diffuser 508 isconfigured to diffuse right-circularly polarized (RCP) light that isincident thereon within a particular range of angles. In such cases, thediffuser 508 may permit light that is left-circularly polarized (LCP)and/or incident on the diffuser 508 at other angles (e.g., 0 degrees) topass through without being diffused, though some attenuation of thelight may occur. Although light having particular handedness ofpolarization and incident on optical elements at particular angle(s) aredescribed herein for illustrative purposes, in other embodiments lightthat has any handedness of polarization and/or is incident on opticalelements within any technically feasible range(s) of angles, may beused. For example, in some embodiments the diffuser 508 may instead beconfigured to diffuse LCP light incident on the diffuser 508 within aparticular range of angles and permit light that is RCP and/or incidenton the diffuser 508 at other angles to pass through without beingdiffused.

As shown, the diffuser 508 is transparent to light from a real-worldscene behind the diffuser 508. As a result, a user (wearing, e.g., a HMDthat includes the foveated display system 400) can observe objects inthe real-world scene in addition to generated content that is diffusedby the diffuser 508 (and generated content that is focused on a fovealregion of the user's eye gaze by the foveal display module 512, asdiscussed in greater detail below). Although optical elements aresometimes described herein as being transparent, it should be understoodthat some attenuation of light may occur as the light passes through“transparent” optical elements.

As shown, the diffused imagery that is produced by the diffuser 508 isfurther passed through a pancake lens 510, which corresponds to thepancake lens 408 and is configured to increase a propagating distance oflight, thereby causing the diffused imagery to appear further away to auser. In some embodiments, the diffuser 508 may be included within thepancake lens 510, rather than being external to the pancake lens 510 asshown in FIG. 5 . Such embodiments can be relatively compact and arediscussed in greater detail below in conjunction with FIGS. 6-7 .

Turning now to the foveal display module 512, as shown, the fovealdisplay module includes a projection device 514 and a spatial lightmodulator (SLM) 516. In some embodiments, the SLM 516 is configured tomodulate light incident thereon to provide a holographic display. Insuch cases, an application (e.g., one of the applications stored in theapplication store 355) or engine (e.g., the engine 365) may determinemodulation(s) to light emitted by the projection device 514 required togenerate virtual imagery and control the SLM 516 accordingly. Inaddition, light modulated by the SLM 516 may be focused onto the MEMSmirror 524 via a concave mirror 520 and one or more beam splitters(e.g., beam splitter 522).

In operation, an eye tracking module 528 is configured to track thepupil position of a user's eye. For example, in some embodiments, anapplication (e.g., one of the applications stored in the applicationstore 355) or engine (e.g., the engine 365) may analyze tracking datarelated to the user's eye that is generated using one or moreillumination sources and an imaging device (camera) included in a NED orHMD that is worn by the user, as described above in conjunction withFIG. 3 .

The MEMS mirror 524 is controllable to steer light that has beenreflected by the concave mirror 520 onto the MEMS mirror 524, based on apupil position detected by the eye tracking module 528. That is, theMEMS mirror 524 may be controlled to perform gaze-following steering. Inparticular, the MEMS mirror 524 may be steered such that high-resolutionimagery is reflected and focused by a HOE 526 so as to pass through afoveal region of the user's eye gaze corresponding to the detected pupilposition. For example, the MEMS mirror 524 could be steered such thatmost light rays reflected by the MEMS mirror 524 pass through the user'spupil to provide appropriate gaze-direction views. In some embodiments,the MEMS mirror 524 may be steerable in three-directions to providedifferent fields of view that follow the user gaze direction.

In some embodiments, the foveal display module 512 may further include aswitchable PBP grating stack that increases a steering range of the MEMSmirror 524, as discussed in greater detail below in conjunction withFIGS. 7-8 . As used herein, the “steering range” of the MEMS mirror 524refers to a range of angles through which the MEMS mirror 524 can steerlight that is incident thereon. It should be understood that thesteering range of the MEMS mirror 524 is limited by the tilt angle ofthe MEMS mirror 524, i.e., the angle at which the MEMS mirror 524 can bemechanically deflected. In some cases, a limited steering range of theMEMS mirror 524 may need to be extended to cover the entire field ofview that is visible to an eye of a user. A switchable PBP grating stackmay be used to increase the steering range of the MEMS mirror 524 insome embodiments. It should be understood, however, that the switchablePBP grating stack may be unnecessary if the MEMS mirror 524 hassufficient steering range for the relevant application.

As described, the HOE 526 is a lens having focal power and is furtherpolarization, angular, and wavelength selective in some embodiments. Insuch cases, the HOE 526 may pass through most light, including light forthe low-resolution, high FOV background imagery generated by theperipheral display module 502, as well as real-world light. At the sametime, the HOE 526 is configured to reflect and focus light from the MEMSmirror 524 having particular wavelengths and incident on the HOE 526within a particular range of angles toward a foveal region of a user'seye gaze. As a result, the user can observe high-resolution virtualimagery that is generated by the foveal display module 512 and focusedonto the foveal region of the user's eye gaze, as well aslow-resolution, large FOV background imagery generated by the peripheraldisplay module 502 and real-world light that has passed through opticalelements of the peripheral display module 502 and the foveal displaymodule 512.

Although specific optical elements and devices are discussed herein asreference examples, in alternative embodiments any othertechnically-feasible types of optical elements and/or devices that arecapable of performing the functionalities of the optical elements and/ordevices disclosed herein may be used. In some embodiments, the foveateddisplay systems 400 and 500 may also include other optical elements thatare not shown. For example, the foveated display system 400 or 500 mayinclude a combiner that is configured to combine real-world light withlight that has been diffused and reflected by the diffuser 406 or thediffuser 510, respectively.

In some embodiments, each of the foveated display systems 400 and 500may be included in a NED or HMD, such as the NEDs 100 or 300, or the HMD162, described above in conjunction with FIGS. 1-3 . In otherembodiments, each of the foveated display systems 400 and 500 mayinclude or be implemented in conjunction with an artificial realitysystem on any technically feasible platform, such as a mobile device, acomputing system, or any other hardware platform capable of providingartificial reality content to one or more users.

Pancake Lens Including Diffuser

FIG. 6 is a schematic diagram illustrating a pancake lens 600 thatincludes a diffuser, according to various embodiments. As described, apancake lens is a folded optic in which light that enters reflects, or“bounces,” through multiple times before exiting. Illustratively, lightbounces twice through the pancake lens 600. The multiple bouncesincrease the propagation distance of light, which can in turn increasethe perceived distance of imagery from a viewer and/or magnify theimagery.

As shown, the pancake lens 600 includes a polarization- andangular-selective mirror 602, a half-wave plate 604, a diffuser 606, anda half mirror 608. All or some of the components of the pancake lens 600may be in physical contact with one another, share a substrate with oneanother, laminated with one another, optically in contact with oneanother, have index matching fluid or optical glue between one another,and/or may have space therebetween. For example, all or some of the someof the components of the pancake lens 600 could be the surfaces oflenses.

The polarization- and angular-selective mirror 602 is configured toselectively reflect one or more handedness of polarization of light thatis incident on the mirror 602 within one or more ranges of angles, whileallowing through other light. In some embodiments, the polarization- andangular-selective mirror 602 may selectively allow through LCP lightthat is incident on the mirror 602 within a particular range of anglesand RCP light that is incident on the mirror 602 at 0°, while reflectingLCP light that is incident on the mirror 602 at 0°, as discussed ingreater detail below in conjunction with FIG. 7 . Further, thepolarization- and angular-selective mirror 602 maintains thepolarization of reflected light, like a Bragg reflector. In someembodiments, the polarization- and angular-selective mirror 602 may be avolume grating mirror constructed from a liquid crystal material havingpolarization selectivity. More generally, the polarization- andangular-selective mirror 602 may be constructed from anytechnically-feasible material(s).

In contrast to the polarization- and angular-selective mirror 602, thehalf mirror 608 is a simple meta-mirror that does not maintain thehandedness of reflected light. In particular, the half mirror 608 isconfigured to allow through one handedness of polarization of lightwhile reflecting the other handedness of polarization. In addition, thehandedness of polarization of light reflected by the half mirror 608 isconverted to the opposite handedness. For example, in some embodiments,the half mirror 608 may allow through LCP light while reflecting RCPlight as LCP light, as discussed in greater detail below in conjunctionwith FIG. 7 .

The half-wave plate 604 is configured to retard one linear component oflight relative to the other component by 180°. As a result, thehalf-wave plate 604 converts the handedness of polarization of lightincident thereon into the other handedness of polarization. For example,the half-wave plate 604 could convert LCP light into RCP light, and viceversa.

The diffuser 606 is similar to the diffusers 406 and 508 described abovein conjunction with FIGS. 4 and 5 , respectively, in some embodiments.In such cases, the diffuser 606 may be polarization, angular, andwavelength selective. For example, the diffuser 606 could be configuredto diffuse RCP light from a projector that is incident thereon within aparticular range of angles while allowing through other light, asdiscussed in greater detail below in conjunction with FIG. 7 .

As shown, the diffuser 606 is disposed between the half-wave plate 604and the half mirror 608 within the pancake lens 600. An optical systemin which the diffuser 606 is included within the pancake lens 600 ismore compact than systems having a diffuser that is external to apancake lens, which can be beneficial for applications with a HMD orother devices where a small form factor and weight are considerations.

FIG. 7 is a ray-tracing diagram illustrating operation of the pancakelens 600 that includes the diffuser 606, according to variousembodiments. As shown, LPC light (from, e.g., the projection device 404or 504) enters the pancake lens 600 at a slanted angle via thepolarization- and angular-selective mirror 602. Illustratively, thepolarization- and angular-selective mirror 602 is configured to allowthrough LCP light that is incident on the mirror 602 within a range ofangles in some embodiments. In addition, the polarization- andangular-selective mirror 602 is configured to allow other light to passthrough.

As shown, the LCP light that is incident on the polarization- andangular-selective mirror 602 at an angle passes through the mirror 602,which is configured to selectively allow through LCP light that isincident on the mirror 602 within a particular range of angles and RCPlight that is incident on the mirror 602 at 0°, while reflecting LCPlight that is incident on the mirror 602 at 0°. The LCP light that haspassed through the polarization- and angular-selective mirror 602 isthen incident on the half-wave plate 604, which converts the LCP lightto RCP light.

As shown, the RCP light produced by the half-wave plate 604 is incidenton the diffuser 606 at a slanted angle. In some embodiments, thediffuser 606 is configured to diffuse or scatter such RCP light that isincident on the diffuser 606 within a range of angles. In addition, thediffuser 606 is configured to permit light that is LCP and/or incidenton the diffuser 606 at other angles (e.g., 0 degrees) to pass throughwithout being diffused, although some attenuation of the light mayoccur.

As shown, diffused light that is produced by the diffuser 606 is RCP andreflects off of the diffuser 606 at 0° while maintaining its handednessof polarization. The reflected RCP light passes through the half-waveplate 604 again, which converts the RCP light to LCP light. The LCPlight is then incident on the polarization- and angular-selective mirror602, which completes one bounce through the pancake lens 600.

A second bounce through the pancake lens 600 begins when the LCP lightis reflected by the polarization- and angular-selective mirror 602mirror 602. As shown, the polarization- and angular-selective mirror 602maintains the handedness of the reflected light, which is LCP. Thereflected LCP light passes through the half-wave plate 604 and isconverted to RCP light. The RCP light then passes through the diffuser606 again, while maintaining its handedness of polarization and notbeing diffused. Thereafter, the RCP light that has passed through thediffuser 606 is reflected by the half mirror 608.

As described, the half mirror 608 is a simple meta-mirror that does notmaintain the handedness of reflected light. As shown, RCP light that isincident on the half mirror 608 is reflected as LCP light. In someembodiments, the half mirror 608 may be a concave half mirror with focalpower that can be used to make imagery appear at a further distance. Asshown, some light passes through the half mirror 608 and is lost to theoutside. However, the percentage of light that is lost may be relativelysmall in embodiments (e.g., approximately one percent).

As shown, the LCP light that is reflected by the half mirror 608 passesthrough the diffuser 606 again, without being diffused, and is thenconverted by the half-wave plate 604 to RCP light. Thereafter, the RCPlight passes through the mirror 602, which is configured to allowthrough such RCP light, and exits the pancake lens 600 towards aneyebox.

As further shown, real-world light that is LCP passes directly throughthe pancake lens 600. In operation, the half mirror 608 separatesreal-world light that is incident thereon into a LCP component that ispassed through the half mirror 608 and a RCP component that is reflectedby the half mirror 608. In some embodiments, a circular polarizer (notshown) may be used to selectively allow through a LCP component ofreal-world light before the LCP component is incident on the half mirror608.

As shown, the LCP real-world light that has passed through the halfmirror 608 passes through the diffuser 606, while maintaining itshandedness of polarization and not being diffused. Thereafter, the LCPreal-world light passes through the half-wave plate and is converted bythe half-wave plate 624 to RCP light. The RCP light then passes throughthe mirror 602 and exits the pancake lens 600 towards an eyebox.

As described, a pancake lens that includes a diffuser is a more compactdesign than one in which the diffuser and the pancake lens are distinct.As a result, a system that requires a diffuser and a pancake lens (e.g.,the peripheral display modules 402 or 502, or the foveated displaysystems 400 and or 500) can have a relatively compact construction. Suchcompactness can be beneficial for applications with a HMD or otherdevices where a small form factor and weight are considerations. Inaddition, the pancake lens 600 may produce relatively little cross-talk,due to the angular selectivity of the diffuser 606, or expected lightreduction.

Although specific optical elements of the pancake lenses 600 arediscussed herein as reference examples, in alternative embodiments anyother technically-feasible types of optical elements that are capable ofperforming the functionalities of the optical elements disclosed hereinmay be used.

Foveal Display System with Switchable PBP Grating Stack

FIG. 8 is a schematic diagram illustrating an optical configuration ofcomponents of the foveal display module 512 of FIG. 5 , according tovarious embodiments. Although described with respect to the fovealdisplay module 512 of FIG. 5 as an illustrative example, the opticalconfiguration of FIG. 8 is also applicable to other foveal displaysystems such as the foveal display module 410 of FIG. 4 .

As shown, the foveal display module 512 includes the SLM 516, theconcave mirror 520, the MEMS mirror 524, and the HOE 526, which aredescribed above in conjunction with FIG. 5 . Although light is shown aspassing through the MEMS mirror 524 for illustrative purposes, it shouldbe understood that light is actually reflected by the MEMS mirror 524.As described, the SLM 516 included in the foveal display system providesa holographic display, and light from the holographic display can befocused on a foveal region of a user's eye gaze. In particular,responsive to detected changes in pupil position corresponding to thefoveal region location, the MEMS mirror 524 can be controlled to steerlight from the holographic display at particular angle(s). Inparticular, the MEMS mirrors 420 may steer most light rays from theholographic display such that those light rays reflect off of the HOE526 and pass through the detected pupil position. As a result, thefoveal display module 512 can generate appropriate gaze-direction views.

As shown, the foveal display module 512 further includes an optionalswitchable PBP grating stack 802. The switchable PBP grating stack 802is disposed before the MEMS mirror 524 in the light path. The switchablePBP grating stack 802 is configured to extend the steering range of theMEMS mirror 524. In some embodiments, the switchable PBP grating stack802 may double the steering range of the MEMS mirror 524. For example,if the horizontal steering range of the MEMS mirror 524 is +/−15°without active PBP grating elements, then the horizontal steering rangecould be extended to +/−30° with active PBP elements. Assuming thevertical steering range is +/−10° without active PBP elements, then theoverall dynamic field of view could be 75°×35°, with a diagonal of 79°.Although described herein for simplicity with respect to a switchablePBP stack that extends the steering range of the MEMS mirror 524 alongone direction (e.g., horizontally), in some embodiments a foveal displaymodule may also include another switchable PBP stack that extends thesteering range of the MEMS mirror 524 along a perpendicular direction(e.g., vertically). Continuing the example from above, a verticalsteering range of +/−10° could be doubled to +/−20° using such aswitchable PBP stack that extends the vertical steering range.

FIG. 9 illustrates in greater detail components of the foveal displaymodule 512, according to various embodiments. Although described withrespect to the foveal display module 512 of FIG. 5 as an illustrativeexample, components shown in FIG. 9 may also be included in other fovealdisplay systems such as the foveal display module 410 of FIG. 4 .

As shown, the foveal display module 512 includes the SLM 516 thatprovides a holographic display. The foveal display module 512 furtherincludes the beam splitter 506 that, together with another beam splitter902 and a concave mirror (not shown), focus light from the SLM 516towards the switchable PBP grating stack 802 and the MEMS mirror 524.

As shown, the switchable PBP grating stack 802 is disposed before theMEMS mirror 524 in the light path. In operation, the switchable PBPgrating stack 802 and the MEMS mirror 524 are used to steer light, viathe HOE 526, towards a pupil position corresponding to the foveal regionof a user's eye gaze, which is shown as region 904 in FIG. 9 . As shown,light is steered by the MEMS mirror 524 towards the HOE 526 at an anglethat is determined based on pupil position (which is tracked by the eyetracker 528 the eye tracker 528). The steered light that is incident onthe HOE 526 is reflected and focused by the HOE 526 towards the pupil904, due to angular- and wavelength-selectivity characteristics of theHOE 526. Due to the same angular and wavelength selectivitycharacteristics, light for low-resolution, high FOV background imagerygenerated by the peripheral display module 502, as well as real-worldlight, may pass through the HOE 526.

FIG. 10 illustrates components and operation of the switchable PBPgrating stack 802 of FIG. 8 , according to various embodiments. As shownin panel A, the switchable PBP grating stack 802 includes two PBPgratings 1002 and 1006, as well as a switchable half-wave plate 1004that is disposed between the PBP gratings 1002 and 1006. All or some ofthe components of the switchable PBP grating stack 802 may be inphysical contact with one another, share a substrate with one another,laminated with one another, optically in contact with one another, haveindex matching fluid or optical glue between one another, and/or mayhave space therebetween. For example, all or some of the some of thecomponents of the PBP grating stack 802 could be the surfaces of lenses.

As discussed in greater detail below with respect to FIG. 11 , each ofthe PBP gratings 1002 and 1006 has three possible states, depending on ahandedness of polarization of light that is incident on the PBP grating1002 or 1006 and the strength of an electric field that is applied (ornot) to the PBP grating 1002 or 1006. The PBP gratings 1002 and 1006diffract light differently in each of the three states. As a result,light that is incident on the PBP gratings 1002 and 1006 can beredirected at different angles depending the states that the PBPgratings 1002 and 1006 are in.

The switchable half-wave plate 1004 has two possible states, an on stateand an off state. When the switchable half-wave plate 1004 is in the offstate, the switchable half-wave plate 1004 allows light to pass throughunimpeded. That is, the switchable half-wave plate 1004 acts like glass,without any characteristics, in the off state. In the on state, theswitchable half-wave plate 1004 acts as a half-wave plate that retardsone linear component of light relative to the other component by 180°.In the on state, the switchable half-wave plate 1004 converts LCP lightthat is incident thereon into RCP light, and vice versa.

As shown in panel A, the PBP gratings 1002 and 1006 are configured suchthat diffraction angles produced by the PBP gratings 1002 and 1006cancel when the switchable half-wave plate 1004 is in the off state. Asa result, the propagating direction of light passing through theswitchable PBP grating stack 802 remains unchanged when the switchablehalf-wave plate 1004 (and the overall switchable PBP stack 802) is inthe off state.

As shown in panel B, the PBP gratings 1002 and 1006 are furtherconfigured such that diffraction angles produced by the PBP gratings1002 and 1006 add together when the switchable half-wave plate 1004 isin the on state. As a result, the propagating direction of light thatpasses through the switchable PBP grating stack 802 is changed when theswitchable half-wave plate 1004 (and the overall switchable PBP stack802) is in the on state.

The difference in diffraction angles between panels A and B is due tothe different states of the PBP grating 1006, which is in turn caused bythe different handedness of polarization of light output by theswitchable half-wave plate 1004 when the switchable half-wave plate 1004is in the on state versus the off state. As a result of the differentdiffraction angles, the switchable PBP grating stack 802 can becontrolled to steer a beam of light incident thereon by turning theswitchable half-wave plate 1004 (and the overall switchable PBP stack802) on or off, as appropriate. Further, the steering range provided bythe switchable PBP grating stack 802 can be used to increase thesteering range of a beam-steering device such as the MEMS mirror 524(by, e.g., a factor of 2), as described above in conjunction with FIGS.8-9 .

Although specific optical elements of the PBP grating stack 800 arediscussed herein as reference examples, in alternative embodiments anyother technically-feasible types of optical elements that are capable ofperforming the functionalities of the optical elements disclosed hereinmay be used.

FIG. 11 illustrates a PBP grating 1100, according to variousembodiments. In some embodiments, the PBP grating 1100 may be one of thePBP gratings 1002 and 1006 described above in conjunction with FIG. 8 .Mutually orthogonal x and y-axes 1110 are illustrated for reference. Thez-axis, not illustrated, is perpendicular to the x-y plane and along anoptical axis of the grating 1100.

As shown, the grating 1100 includes uniaxial fast axis 1120 of LC ormeta structure that are oriented in a linearly repetitive pattern. InFIG. 11 , the orientations of the fast axes are illustrated as shortline segments aligned so as to schematically represent orientations ofthe LCs or the meta structure. For example, the fast axis 1120A isoriented in the x-direction while LC 1120B is oriented in they-direction. A fast axis between 1120A and 1120B are aligned alongdirections intermediate to the x and y-directions. The uniaxialwaveplate having such a patterned orientation gives rise to ageometric-phase shift of light as a consequence of polarizationevolution as light waves of the light propagate through the waveplate(e.g., phase plate). In various embodiments, orientations of the fastaxis along the x-axis are constant for a particular x-y plane of thegrating 1100. Further, though not illustrated, in various embodiments,orientations of the fast axis in a direction perpendicular to the x-yplane (the z-axis) may vary in a rotational fashion (e.g., a twistedstructure).

The linearly repetitive pattern of the grating 1100 has a pitch that ishalf the distance 1130 along the y-axis between repeated portions of thepattern. The pitch determines, in part, the optical properties of thegrating 1100. For example, polarized light incident along the opticalaxis on the grating 1100 results in a grating output comprising primary,conjugate, and leakage light respectively corresponding to diffractionorders m=+1, −1, and zero. Although m=+1 is herein considered to be theprimary order and the conjugate order is considered to be the m=−1order, the designation of the orders could be reversed or otherwisechanged. The pitch determines the diffraction angles (e.g.,beam-steering angles) of the light in the different diffraction orders.Generally, the smaller the pitch, the larger the angles for a givenwavelength of light.

In some embodiments, a PBP grating, such as 1100, may be active (alsoreferred to as an “active element”) or passive (also referred to as a“passive element”). An active PBP grating, for example, has threeoptical states, similar to that of an active PBP lens: an additivestate, a neutral state, and a subtractive state. In an additive state,the active PBP grating diffracts light of a particular wavelength to anangle that is positive relative to the diffraction angle of thesubtractive state. In the subtractive state, the active PBP gratingdiffracts light at a particular wavelength to an angle that is negativerelative to the positive angle of the additive state. On the other hand,in the neutral state, the PBP grating does not lead to a diffraction oflight and does not affect the polarization of light passing through theactive PBP grating.

The state of an active PBP grating may be determined by a handedness ofpolarization of light incident on the active PBP grating and a measureof the electric field applied to the active PBP grating. For example, insome embodiments, an active PBP grating operates in a subtractive stateresponsive to incident light with a right-handed circular polarizationand an applied electric field of zero (or, more generally, below athreshold electric field). In some embodiments, the PBP grating operatesin an additive state responsive to incident light with a left-handedcircular polarization and an applied electric field of zero. In someembodiments, the PBP grating operates in a neutral state (regardless ofpolarization) responsive to an applied electric field. Liquid crystalswith positive dielectric anisotropy may be aligned along an appliedelectric field direction. If the active PBP grating is in the additiveor subtractive state, then light output from the active PBP grating hasa handedness that is opposite the handedness of light input into theactive PBP grating. If the active PBP grating is in the neutral state,then light output from the active PBP grating has the same handedness asthe light input into the active PBP grating.

The state of a passive PBP grating is determined by a handedness ofpolarization of light incident on the passive PBP grating. For example,in some embodiments, a passive PBP grating operates in a subtractivestate responsive to incident light with a right-handed circularpolarization. In some embodiments, the passive PBP grating operates inan additive state responsive to incident light with a left-handedcircular polarization. For the passive PBP grating in the additive orsubtractive state, light output from the passive PBP grating has ahandedness that is opposite the handedness of light input into thepassive PBP grating.

Generating Artificial Reality Content Using a Foveated Display System

FIG. 12 illustrates a method for generating artificial reality contentusing a foveated display system, according to various embodiments.Although the method steps are described with reference to the systems ofFIGS. 1-3 and 5-11 , persons skilled in the art will understand that anysystem may be configured to implement the method steps, in any order, inother embodiments. In particular, the method is described with referenceto the foveated display system 500 of FIG. 5 as an illustrative example,but the method may also be implemented with other foveated displaysystems such as the foveated display system 400 of FIG. 4 .

As shown, a method 1200 begins at block 1202, where an applicationcauses a projected image to be generated using at least the diffuser 508that is in line with the HOE 526. The application may be, e.g., one ofthe applications stored in the application store 355, which as describedabove in conjunction with FIG. 3 may include gaming applications,conferencing applications, video playback applications, or any othersuitable applications. Although described with respect to such anapplication, in other embodiments some or all steps of the method 1200may be performed by an engine such as the engine 365 described above inconjunction with FIG. 3 .

As described above in conjunction with FIGS. 4-7 , the diffuser 508 maybe polarization, angular, and wavelength selective in some embodiments.In such cases, diffuser 508 may reflect and diffuse light having aparticular handedness of polarization and within a particular range ofwavelengths that is projected onto the diffuser 508 within a particularrange of angles, while allowing other light (e.g., real-world light) topass through the diffuser 508. As a result, the diffuser 508 may be usedto generate a low-resolution, large FOV background image for non-fovealregions of a user's eye gaze. In some embodiments, the diffuser 508 maybe included within a pancake lens, as described above in conjunctionwith FIGS. 6-7 .

FIG. 13 illustrates in greater detail block 1202 of the method 1200 ofFIG. 12 , according to various embodiments. Although the method stepsare described with reference to the systems of FIGS. 5-7 , personsskilled in the art will understand that any system may be configured toimplement the method steps, in any order, in other embodiments. Inparticular, although described with reference to the foveated displaysystem 500 of FIG. 5 as an illustrative example, the method steps mayalso be implemented with other foveated display systems such as thefoveated display system 400 of FIG. 4 .

At block 1302, the application determines a background image for thenon-foveal regions of a user's eye gaze. As described, the non-fovealregions correspond to the peripheral vision of the user, so thebackground image that is determined at block 1304 need not be of thesame resolution (i.e., pixel density) as a virtual image generated for afoveal region of the user's eye gaze. The background image that isdetermined at step 1302 may be any suitable image, and the particularimage will generally depend on the application.

At block 1304, the application causes the background image to beprojected at an angle onto the diffuser 508. In some embodiments, thediffuser may be included within a pancake lens that increases apropagating distance of light. In other embodiments, the diffuser may bedistinct from a pancake lens. As described, the diffuser may also bepolarization, angular, and wavelength selective in some embodiments soas to reflect and diffuse light having a particular handedness ofpolarization and particular wavelengths that is projected at particularangles onto the diffuser, while allowing other light (e.g., real-worldlight) to pass through the diffuser.

Returning to FIG. 12 , at block 1204, the application determines a pupilposition of an eye of the user. In embodiments, any technically feasibleeye-tracking technique(s) may be employed. For example, the applicationmay analyze tracking data related to the user's eye that is generatedusing one or more illumination sources and an imaging device (camera)included in a NED or HMD that is worn by the user, as described above inconjunction with FIG. 3 .

At block 1206, the application causes a virtual image to be generatedand focused on a foveal region of a user's eye gaze using at least theHOE 526, based on the pupil position determined at block 1204. In someembodiments, a steering range of the MEMS mirror 524 that is used tofocus light generated via the SLM 516 may further be increased using theswitchable PBP grating stack 802, as described above in conjunction withFIGS. 7-8 .

As described, the HOE 526 is an angular- and wavelength-selective lensthat permits light associated with the projected image and generated atblock 1202, as well as real-world light, to pass through the HOE 526. Inaddition, the HOE 526 is configured to reflect and focus light ofparticular wavelengths from the MEMS mirror 524 that is incident on theHOE 526 at particular angles towards a foveal region of a user's eyegaze, thereby generating high-resolution imagery.

FIG. 14 illustrates in greater detail block 1206 of the method 1200 ofFIG. 12 , according to various embodiments. Although the method stepsare described with reference to the systems of FIGS. 5 and 8-11 ,persons skilled in the art will understand that any system may beconfigured to implement the method steps, in any order, in otherembodiments. In particular, although described with reference to thefoveated display system 500 of FIG. 5 as an illustrative example, themethod steps may also be implemented with other foveated display systemssuch as the foveated display system 400 of FIG. 4 .

As shown, at block 1402, the application causes the SLM 516 to modulatelight emitted by the projection device 514. As described, in someembodiments, the SLM 516 may provide a holographic display that useslight diffraction to create a high-resolution virtual image that isfocused on the foveal region of a user's eye gaze. In such cases, theapplication may determine modulation(s) to light emitted by theprojection device 514 that are required to generate the high-resolutionvirtual imagery and control the SLM 516 accordingly. Any suitablemodulations may be determined, and the particular modulations willgenerally depend on the application. In addition, light modulated by theSLM 516 may be focused onto the MEMS mirror 524 via the concave mirror508 and one or more beam splitters (e.g., the beam splitters 506 and902) in some embodiments.

As shown, at block 1404, the application determines an angle with whichto steer light using the MEMS mirror 414 onto the HOE 526 based on thepupil position determined at block 1204 of the method 1200. Asdescribed, the determined angle is an angle necessary to focus lightonto a foveal region of a user's eye gaze that corresponds to thedetermined pupil position.

At block 1406, the application determines whether the angle is within asteering range of the MEMS mirror 524. As described, a steering range ofthe MEMS mirror 524 may be limited by the tilt angle achievable by theMEMS mirror 524 in some embodiments. For example, a horizontal steeringrange of the MEMS mirror 524 could be limited to +/−15° without activePBP grating elements, and a vertical steering range of the MEMS mirror524 could be limited to +/−10° without active PBP elements.

If the application determines that the angle is within the steeringrange of the MEMS mirror 524, then at block 1408, the application causesthe MEMS mirror to steer light at the angle determined at block 1404.For example, the application could send a control signal toelectromagnetically drive the MEMS mirror 524 based on the angledetermined at block 1404. In this case, the switchable PBP grating stack802 is not switched on (or is switched off if the switchable PBP gratingstack 802 is already on).

If, on the other hand, the angle is not within the steering range of theMEMS mirror 524, then at block 1410, the application determines an anglewith which to steer light onto the HOE 526 using the MEMS mirror 524with the switchable PBP grating stack 802 switched on. For example, ifthe switchable PBP grating stack 802 is configured to double thesteering range of the MEMS mirror 524, then the application could dividethe angle determined at block 1404 by two at block 1410.

At block 1412, the application causes the switchable PBP grating stack802 to be switched on and the MEMS mirror 524 to steer light at theangle determined at block 1410. As described, the switchable PBP gratingstack 802 may be switched on by switching on the switchable half-waveplate 1004.

Returning to FIG. 12 , at step 1208, the application determines whetherto continue to another point in time. If the application determines tocontinue, then the method 1200 returns to step 1202, where theapplication causes another projected image to be generated using atleast the diffuser 508 that is in line with the HOE 526. On the otherhand, if the application determines not to continue, then the method1200 ends.

One advantage of the foveated display systems disclosed herein is thatthe foveated display systems generate high-resolution virtual imageryfor a foveal region of a user's eye gaze along with low-resolution,large field of view background imagery for other regions of the user'seye gaze. A diffuser that is used to generate the projected imagery canbe disposed within a pancake lens, which is a relatively compact (i.e.,thinner) design that is beneficial for applications with a HMD or otherdevices where a small form factor and weight are considerations. Inaddition, a switchable Pancharatnam-Berry phase grating stack can beused to increase the steering range of a beam-steering device used togenerate the high-resolution virtual imagery such that, e.g., lightassociated with the virtual imagery can be steered to cover an entirefield of view that is visible to the user's eye. These technicaladvantages represent one or more technological advancements over priorart approaches.

1. Some embodiments include an optical system comprising twoPancharatnam-Berry Phase (PBP) gratings, and a switchable half-waveplate disposed between the PBP gratings.

2. The optical system according to clause 1, wherein the PBP gratingsare configured to diffract light at an angle when the switchablehalf-wave plate is switched on, and the PBP gratings are configured topass through light when the switchable half-wave plate is switched off.

3. The optical system according to clauses 1 or 2, further comprising aneye tracking module, wherein the switchable half-wave plate iscontrolled based on a pupil position determined using the first eyetracking module.

4. The optical system according to any of clauses 1-3, furthercomprising two additional PBP gratings and an additional switchablehalf-wave plate disposed between the two additional PBP gratings.

5. The optical system according to any of clauses 1-4, wherein the twoPBP gratings and the switchable half-wave plate are configured to steerlight in a first direction, and wherein the two additional PBP gratingsand the additional switchable half-wave plate are configured to steerlight in a second direction that is perpendicular to the firstdirection.

6. The optical system according to any of clauses 1-5, furthercomprising a beam-steering device, wherein the two PBP gratings and theswitchable half-wave plate are used to increase a steering range of thebeam-steering device.

7. The optical system according to any of clauses 1-6, wherein the twoPBP gratings and the switchable half-wave plate are disposed in a lightpath before the beam-steering device.

8. The optical system according to any of clauses 1-7, wherein theoptical system is included in a foveal display module of a head-mounteddisplay configured to focus imagery on a foveal region of an eye gaze ofa user.

9. The optical system according to any of clauses 1-8, wherein theswitchable half-wave plate passes through light in an off state andchanges a handedness of polarization of light in an on state.

10. Some embodiments include a display system comprising a light source,and an optical stack comprising a plurality of Pancharatnam-Berry Phase(PBP) gratings, wherein the optical stack is switchable between at leasttwo modes.

11. The display system according to clause 10, wherein the at least twomodes include a first mode in which light incident on the optical stackpasses through the optical stack and a second mode in which lightincident on the optical stack is diffracted at an angle by the opticalstack.

12. The display system according to clauses 10 or 11, wherein theoptical stack comprises two PBP gratings and a switchable half-waveplate disposed between the two PBP gratings.

13. The display system according to any of clauses 10-12, wherein theoptical stack further comprises two additional PBP gratings and anadditional switchable half-wave plate disposed between the twoadditional PBP gratings.

14. The display system according to any of clauses 10-13, wherein thetwo PBP gratings and the switchable half-wave plate are used to steerlight in a first direction, and wherein the two additional PBP gratingsand the additional half-wave plate are used to steer light in a seconddirection that is perpendicular to the first direction.

15. The display system according to any of clauses 10-11, furthercomprising a microelectro-mechanical system (MEMS) mirror, wherein theoptical stack is controllable to increase a steering range of the MEMSmirror.

16. The display system according to any of clauses 10-15, wherein theoptical stack is disposed in a light path before the MEMS mirror.

17. The display system according to any of clauses 10-16, wherein thedisplay system is included in an artificial reality system.

18. Some embodiments include a method comprising detecting a pupilposition of an eye of a user, determining an angle to steer light basedon the detected pupil position, and steering the light at the angleusing at least an optical stack comprising a plurality ofPancharatnam-Berry Phase (PBP) gratings, wherein the optical stack isswitchable between at least two modes.

19. The method according to clause 18, wherein the optical stackcomprises two PBP gratings and a switchable half-wave plate disposedbetween the two PBP gratings, and the at least two modes include a firstmode in which light incident on the optical stack passes through theoptical stack and a second mode in which light incident on the opticalstack is diffracted at an angle by the optical stack.

20. The method according to clause 18 or 19, wherein the light isfurther steered at the angle using a microelectro-mechanical system(MEMS) mirror, and wherein the optical stack is controllable to increasea steering range of the MEMS mirror.

Any and all combinations of any of the claim elements recited in any ofthe claims and/or any elements described in this application, in anyfashion, fall within the contemplated scope of the present disclosureand protection.

The foregoing description of the embodiments of the disclosure has beenpresented for the purpose of illustration; it is not intended to beexhaustive or to limit the disclosure to the precise forms disclosed.Persons skilled in the relevant art can appreciate that manymodifications and variations are possible in light of the abovedisclosure.

Some portions of this description describe the embodiments of thedisclosure in terms of algorithms and symbolic representations ofoperations on information. These algorithmic descriptions andrepresentations are commonly used by those skilled in the dataprocessing arts to convey the substance of their work effectively toothers skilled in the art. These operations, while describedfunctionally, computationally, or logically, are understood to beimplemented by computer programs or equivalent electrical circuits,microcode, or the like. Furthermore, it has also proven convenient attimes, to refer to these arrangements of operations as modules, withoutloss of generality. The described operations and their associatedmodules may be embodied in software, firmware, hardware, or anycombinations thereof.

Any of the steps, operations, or processes described herein may beperformed or implemented with one or more hardware or software modules,alone or in combination with other devices. In one embodiment, asoftware module is implemented with a computer program productcomprising a computer-readable medium containing computer program code,which can be executed by a computer processor for performing any or allof the steps, operations, or processes described.

Embodiments of the disclosure may also relate to an apparatus forperforming the operations herein. This apparatus may be speciallyconstructed for the required purposes, and/or it may comprise ageneral-purpose computing device selectively activated or reconfiguredby a computer program stored in the computer. Such a computer programmay be stored in a non-transitory, tangible computer readable storagemedium, or any type of media suitable for storing electronicinstructions, which may be coupled to a computer system bus.Furthermore, any computing systems referred to in the specification mayinclude a single processor or may be architectures employing multipleprocessor designs for increased computing capability.

Embodiments of the disclosure may also relate to a product that isproduced by a computing process described herein. Such a product maycomprise information resulting from a computing process, where theinformation is stored on a non-transitory, tangible computer readablestorage medium and may include any embodiment of a computer programproduct or other data combination described herein.

Finally, the language used in the specification has been principallyselected for readability and instructional purposes, and it may not havebeen selected to delineate or circumscribe the inventive subject matter.It is therefore intended that the scope of the disclosure be limited notby this detailed description, but rather by any claims that issue on anapplication based hereon. Accordingly, the disclosure of the embodimentsis intended to be illustrative, but not limiting, of the scope of thedisclosure, which is set forth in the following claims.

The descriptions of the various embodiments have been presented forpurposes of illustration, but are not intended to be exhaustive orlimited to the embodiments disclosed. Many modifications and variationsis apparent to those of ordinary skill in the art without departing fromthe scope and spirit of the described embodiments.

Aspects of the present embodiments may be embodied as a system, method,or computer program product. Accordingly, aspects of the presentdisclosure may take the form of an entirely hardware embodiment, anentirely software embodiment (including firmware, resident software,micro-code, etc.) or an embodiment combining software and hardwareaspects that may all generally be referred to herein as a “module” or“system.” Furthermore, aspects of the present disclosure may take theform of a computer program product embodied in one or more computerreadable medium(s) having computer readable program code embodiedthereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain, or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

Aspects of the present disclosure are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of thedisclosure. It is understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine. The instructions, when executed via the processor ofthe computer or other programmable data processing apparatus, enable theimplementation of the functions/acts specified in the flowchart and/orblock diagram block or blocks. Such processors may be, withoutlimitation, general purpose processors, special-purpose processors,application-specific processors, or field-programmable gate arrays.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present disclosure. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

While the preceding is directed to embodiments of the presentdisclosure, other and further embodiments of the disclosure may bedevised without departing from the basic scope thereof, and the scopethereof is determined by the claims that follow.

What is claimed is:
 1. An optical system comprising: twoPancharatnam-Berry Phase (PBP) gratings; a switchable half-wave platedisposed between the PBP gratings; and a beam-steering mirror device,wherein the PBP gratings and the switchable half-wave plate are disposedin a light path before the beam-steering device, and the PBP gratingsand the switchable half-wave plate are controllable to increase asteering range of the beam-steering mirror device.
 2. The optical systemof claim 1, wherein: the PBP gratings are configured to diffract lightat an angle when the switchable half-wave plate is switched on; and thePBP gratings are configured to pass through light when the switchablehalf-wave plate is switched off.
 3. The optical system of claim 1,further comprising an eye tracking module, wherein the switchablehalf-wave plate is controlled based on a pupil position determined usingthe eye tracking module.
 4. The optical system of claim 1, furthercomprising two additional PBP gratings and an additional switchablehalf-wave plate disposed between the two additional PBP gratings.
 5. Theoptical system of claim 4, wherein the two PBP gratings and theswitchable half-wave plate are configured to steer light in a firstdirection, and wherein the two additional PBP gratings and theadditional switchable half-wave plate are configured to steer light in asecond direction that is perpendicular to the first direction.
 6. Theoptical system of claim 1, wherein the optical system is included in afoveal display module of a head-mounted display configured to focusimagery on a foveal region of an eye gaze of a user.
 7. The opticalsystem of claim 1, wherein the switchable half-wave plate passes throughlight in an off state and changes a handedness of polarization of lightin an on state.
 8. A display system, comprising: a light source; anoptical stack comprising a plurality of Pancharatnam-Berry Phase (PBP)gratings, wherein the optical stack is switchable between at least twomodes; and a beam-steering mirror device, wherein the optical stack isdisposed in a light path before the beam-steering mirror device, and theoptical stack is controllable to increase a steering range of thebeam-steering mirror device.
 9. The display system of claim 8, whereinthe at least two modes include a first mode in which light incident onthe optical stack passes through the optical stack and a second mode inwhich light incident on the optical stack is diffracted at an angle bythe optical stack.
 10. The display system of claim 8, wherein theoptical stack comprises two PBP gratings and a switchable half-waveplate disposed between the two PBP gratings.
 11. The display system ofclaim 10, wherein the optical stack further comprises two additional PBPgratings and an additional switchable half-wave plate disposed betweenthe two additional PBP gratings.
 12. The display system of claim 11,wherein the two PBP gratings and the switchable half-wave plate are usedto steer light in a first direction, and wherein the two additional PBPgratings and the additional half-wave plate are used to steer light in asecond direction that is perpendicular to the first direction.
 13. Thedisplay system of claim 8, wherein the beam-steering mirror devicecomprises a microelectro-mechanical system (MEMS) mirror.
 14. Thedisplay system of claim 8, wherein the display system is included in anartificial reality system.
 15. A method, comprising: detecting a pupilposition of an eye of a user; determining an angle to steer light basedon the detected pupil position; and steering the light at the angleusing at least a beam-steering mirror device and an optical stackcomprising a plurality of Pancharatnam-Berry Phase (PBP) gratings,wherein the optical stack is switchable between at least two modes, theoptical stack is disposed in a light path before the beam-steeringmirror device, and the optical stack is controllable to increase asteering range of the beam-steering mirror device.
 16. The method ofclaim 15, wherein: the optical stack comprises two PBP gratings and aswitchable half-wave plate disposed between the two PBP gratings; andthe at least two modes include a first mode in which light incident onthe optical stack passes through the optical stack and a second mode inwhich light incident on the optical stack is diffracted at an angle bythe optical stack.
 17. The method of claim 15, wherein the beam steeringmirror device is a microelectro-mechanical system (MEMS) mirror.